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ODP-83-7029 
23 February 1983 



MEMORANDUM FOR: Chairman, Publications Review Board 


VIA: 


FROM: 


SUBJECT : 


Deputy Director for Administration 
Director of Data Processing 

i ^ 

Executive Officer 
Office of Data Processing 

Request to Give a Presentation 


STAT 


1. Chief of Systems Support Division in 

ODP has been invited to make a presentation on 25 March 1983 to 
the Third National Symposium on EDP Quality Assurance, sponsored 
bv the D ata Processing Management Association (DPMA) . Mr. 

presentation is on "A Practical Application Using Risk 


Analysis . " 


2. When was Chief of Quality Assurance Division 

in ODP, his division developed and applied a risk analysis 
questionnaire to a portfolio of projects in ODP Applications. 
The presentation focuses on the development and application of 
the risks questionnaire rather than on the projects on which it 
was applied. 


ILLEGIB 

STAT 


STAT 


STAT 


3. Attached are: a biographical sketch of 


an 


STAT 


outline and corresponding draft copies of viewgraph slides of his 
presentation, a copy of the Risk Questionnaire, and a copy of the 
brochure for the conference, 
fees will be paid by DPMA. 
unclassified. 


conference registration STAT 


His presentation is entirely 


4. 


requests approval to attend the conference and 
make a prese ntation on "A Practical Application Using Risk 
Analysis" . 


ILLEGIB 
STAT 

ILLEGIB 

STAT 
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SUBJECT: Request to Give a Presentation 


AUTHOR'S NAME: 


TITLE OF PRESENTATION: A Practical Application Using Risk 


STAT 


I have reviewed the material attached to this request, 
have found it to the best of my knowledge to be unclassified 
and approve it for presentation at the conference 
been requested to attend. 


and 


has 


STAT 


STAT 


Date 



£igned; H. McDonald 

arry E. Fitzwater , DDA 


Attachments : 

A - Biographical Sketch 
B - Outline 
C - Copy of Slides 
D - Risk Questionnaire 
E - Conf e renc e Brochu re 
r - 9i<\ 


. i mum 

Date 


ODP/A/SSD 


(23Feb83) 


Distribution: 

Original - Addressee (w/atts) 

1 - DD/A Chrono (w/o atts) 

1 - SSD Chrono (w/atts) 

4 - ODP Registry (w/o atts) 
- DDA (w/o arfT 

- D/ODP (lfi/attj) Chrono & 

i ffli/m/iaiHMi (ifluiLu) 

1 - C/EAB/SSD/OS (w/atta), 

r i ~ l 







ubj :P)R:Pubs&PresentApproval 


~r 


r resen 


STAT 


ILLEGIB 


Sanitized Copy Approved for Release 2010/06/07 : CIA-RDP85-00142R000300200005-4 


tr-r 







STAT 


Sanitized Copy Approved for Release 2010/06/07 : CIA-RDP85-00142R000300200005-4 



Sanitized Copy Approved for Release 2010/06/07 : CIA-RDP85-00142R000300200005-4 



Sanitized Copy Approved for Release 2010/06/07 : CIA-RDP85-00142R000300200005-4 


A Practical Application Using Risk Analysis 

I. Take as the definition of project risk: 

Project Risk (high/low) is some measure of expectation 
(high/low) that the critical requirements of a project 
will fail to be met. 

II. Identifying Risk Factors: 

A. McFarlan article in the Harvard Business Review 

B. Project characterization 

III. Risk Questionnaire: 

A. Generation of questions 

B. Weighting of questions 

IV. Results of Questionnaire: 

A. High Risk 

B. Low Risk 

V. Application of Results of Questionnaire: 
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A Practical Application Using Risk Analysis 


Presentation to: 


Third National Symposium 
On EDP Quality Assurance 


25 March 1983 
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Project RISK (high/low) is same measure of expectations 
that the crtical requirements of a project will fail to 


(high/low) 
be met. 
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Identifying Risk Factors 

o McFarlan article 
o Project Characterization 
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Risk Questionnaire 


o Generation of questions 
o Weighting of questions 
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Results of Questionnaire 
o High Risk 
o Low Risk 
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Application of Results of Questionnaire 
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R I SK ASSESSMENT 
QUESTIONNAIRE 
for 

APPLICATION SOFTWARE PROJECTS 



Prepared By: 

Ver i f i cat i on & Validation Branch 
Quality Assurance Division 
Office of Data Processing 
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PROJECT RISK ASSESSMENT QUESTIONNAIRE 

Project: Date: 

PRISM Project Number: Div/Branch: 

Preparer: Phone: 

Reviewer: Phone: 

Current System Status ( check one ) : 

1. System Initiation Phase 

- Preliminary System Requirements Development 

- Detailed System Requirements Development 

2. System Definition Phase 

3. System Design Phase 

- Preliminary System Design 

- Detailed System Design 

4. System Implementation and Integration Phase 

5. System Operations and Support Phase 

- Major system change in progress 

- Minor system change in progress 

- Inactive 
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SIZE RISK ASSESSMENT 


WEIGHT 


Total development person-hours for system 
deve 1 opment / enhancement 

100 to 5,000 Low - 1 

5 , 000 to 25 , 000 Medium - 2 

More than 25,000 High - 3 


2. What is estimated project implementation (F0C) time? 

12 months. or less Low - 1 

13 months to 24 months Medium - 2 

More than 24 months, with phased High - 3 

implementation ( IOC to FOC ) 


More than 24 months, no phasing 


High - 4 


3. Can the project be successfully 
completed within schedule? 


Highly likely 

Low - 1 

Success is likely, or unable to 
estimate 

Medium - 2 

Somewhat doubtful 

High - 3 

Highly unlikely 

High - 4 

What is the project funding ? 


ODP controls funding 

Low - 1 

Joint ODP/User funding control 

Medium - 2 

Major multi-level program funding level 
required 

High - 3 


3 


2 
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SIZE RISK ASSESSMENT 


WEIGHT 


How are the testing resources allocated to the system 
development cycle ? 


Greater than 40% 


Low - 1 


20% to 40% 


Medium - 2 


Less than 20% 


High - 3 


2 


6. Number of logical data groupings which are interrelated 1 


(estimate if unknown) 

Less than 4 Lew - 1 
4 to 6 Medium - 2 
More than 6 High - 3 

7. How many transaction types are projected? 1 

Less than 6 Low - 1 
6 to 25 Medium - 2 
More than 25 High - 3 

8. How many output reports are projected? 1 

Less than 10 Low - 1 
10 to 20 Medium - 2 
More than 10 High - 3 
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STRUCTURE RISK ASSESSMENT WEIGHT 

1. Age of existing automated system 3 

(since last major change) 


Over 2 years 

1 to 2 years, or unknown 
Less than 1 year 

N/A, i.e., no existing automated system 


Low - 1 
Medium - 2 
High - 3 
High - 3 


2. Frequency of change to proposed/ existing system 
(Form 930/Applications Work Order) 

N/A; no existing automated system N/A - 0 

or sufficient development effort 
underway on which to base estimate 

Less than 2 per year Low - 1 

2 to 10 per year Medium - 2 


More than 20 per year 


High - 3 


3. Extent of total system changes in last year 
N/A; no changes 

Affecting less than 10% of programs 
Affecting 10% to 25% of programs 
Affecting more than 25% of programs 


N/A - 0 
Low - 1 
Medium - 2 
High - 3 


3 


3 
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STRUCTURE RISK ASSESSMENT 

4. Severity of system change to be performed 
N/A; new development 
Minor change (s) 

Significant but manageable change 

Major changes in regard to system 
functionality and/or resource 
needs to accomplish change 


WEIGHT 

3 

N/A - 0 
Low - 1 
Medium - 2 
High - 4 


5. Project performance site 
Government facility 
Local, non-government facility 
Not in local area 


Low - 1 
Medium - 2 
High - 5 


2 


6. Staffing of the project (critical staff) 


In - hous e ( government ) 

Low 

Contractor , sole-source 

Medium 

Contractor, competitive bid 

High 


2 


7. What is the type of project organization ? 3 

Line and staff; project has total Low - 1 

management control of development 

personnel 

Mixture of line and staff with Medium - 2 

matrix-managed elements 

Matrix; no management control transferred High - 3 

to project 
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STRUCTURE RISK ASSESSMENT WEIGHT 


8. Is a subcontractor relationship a potential 5 

problem in a contracted effort ? 

N/A; question not applicable to this N/A - 0 

project 

Subcontractor not assigned to an isolated Low - 1 

or critical task; prime contractor has 
previously managed subcontractor successfully 

Subcontractor assigned to all development Medium - 2 
tasks in a subordinate role to prime 
contractor; ODP has favorable experience 
with subcontractor on other effort (s) 

Subcontractor has sole responsibility for High - 3 

critical task; subcontractor new to Agency 
environment 


9. What is the status of the project team 2 
training plan ? 

N/A; no training plan required N/A - 0 
Complete plan in place Low - 1 
Plan under development Medium - 2 
No plan available High - 3 


10. What is the level of skill 3 

used to train project team ? 

N/A; no training required N/A - 0 

Knowledgeable on all systems Low - 1 

Knowledgeable on major components Medium - 2 

Few components understood High - 3 
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STRUCTURE RISK ASSESSMENT WEIGHT 

11. How accessable are supporting reference 3 

and/or compliance documents /information on 
proposed/existing system? 

Readily available Low - 1 

Details available with some Medium - 2 

difficulty and delay 

Great difficulty in obtaining details. High - 3 

except with much delay 


12 . 


What is the availability of documentation 
for the current system ( manual or automated 

Complete and current 

More than 75% complete and current 

Major system and applications 
undocumented or outdated 


) ? 

Low - 1 
Medium - 2 
High - 6 


3 


13. What is the nature of Periodic Maintenance 
support with respect to updating project 
documentation ? 


N/A; new development project 
Close coordination 
Significant but manageable 
Major changes with poor coordination 


N/A - 0 
Low - 1 
Medium - 2 
High - 5 


3 
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STRUCTURE RISK ASSESSMENT WEIGHT 


14. How well does documentation reflect specification/ 3 

program changes? 

N/A; new development project N/A - 0 

Audit trail excellent; good maintenance and Low - 1 
availability of documentation 

Audit trail good; some problems with Medium - 2 

maintenance and availability 

Poor audit trail, inadequate for proper High - 3 

maintenance and availability 


15. What is the documentation approach for the 3 

proposed/existing system? 

Excellent standards closely adhered to Low - 1 

and carried out as integral part of 
system and program development 

Adequate practices but not uniformly Medium - 2 

adhered to 

Poor or no standards; where standards; High - 3 

exist, minimal adherence 


16. What is the approach to development and 
production library control? 

Excellent standards, closely adhered to Low - 1 

Adequate practices, but not uniformly Medium - 2 

adhered to 

Poor or no standards; where standards High - 3 

exist, minimal adherence 


3 


17. What special test facilities are 
available for subsystem testing ? 

Complete or not required 

Limited 

None available 


2 


Low - 1 
Medium - 2 
High - 3 
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STRUCTURE RISK ASSESSMENT 

18. What is the status of the project life 
cycle planning ? 

Current and complete plan 

Plan under development 

No plan present 


WEIGHT 

2 


Low - 1 
Medium - 2 
High - 3 


19. What contingency plans are in place 

to support the operational mission should 
the development /enhancement not be completed 
on schedule ? 

N/A; none required N/A - 0 

Complete plan Lo W - 1 

Major subsystems addressed Medium - 2 

None available High - 3 


20. What is the availability of support 
for the test teams ? 

In place and current 

Only planned 

Major omissions or unplanned 


Low - 1 
Medium - 2 
High - 3 


21. User approval of specifications 

Formal, written approval based on 
structured, detailed review processes 

Formal, written approval based on 
informal, unstructured, detailed 
review processes 

No formal approval; cursory 
review 


Low - 1 


Medium - 2 


High - 3 


/ 
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STRUCTURE RISK ASSESSMENT 


WEIGHT 


22. How much is the development impacted by 5 

external systems? 

N/A; no external systems involved N/A - 0 

All critical inter-system communications Low - 1 

controlled through Interface Control 
Documents; standard protocols utilized; 
interfaces are stable 

All critical inter-system communications Medium - 2 

controlled through Interface Control 

Documents;* some protocols may be 

non-standard; interfaces change 

infrequently 

Not all critical inter-system High - 3 

communications are controlled through 

Interface Control Documents; some protocols 

may be non-standard; 

some interfaces change frequently 


23. What is the type and adequacy of the 
Configuration Management Planning ? 

Complete and functioning 

Undergoing revisions for inadequacies 

None available 


2 


Low - 1 
Medium - 2 
High - 3 
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STRUCTURE RISK ASSESSMENT WEIGHT 


24. Are the development standards and guidelines 4 

realistic and state-of-the-art? 

N/A; in total compliance with 0DP N/A - 0 

standards 

The standards employ structured programming Low - 1 
concepts, reflect current methodology 
and permit tailoring to the nature and 
scope of the development project 

The standards require a top-down Medium - 2 

approach and offer some flexibility in 

application 

The standards are out-of-date and require High - 3 

the application of all aspects ( of standards ) 
to the development project 

25. Is a baseline control process integral 5 

to the overall development discipline? 

N/A; in total compliance with ODP N/A - 0 

standards 

A formal, hierarchical baseline structure is Low - 1 
required; and each baseline, once approved, 
is placed under configuration management 

An informal baseline structure is utilized; Medium - 2 
Minimal configuration control is applied 

No baseline control mechanism is required High - 3 
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STRUCTURE RISK ASSESSMENT WEIGHT 


26. Is the development/enhancement based on 5 

well -specif ied , stable requirements? 

The requirements documentation contains Low - 1 

detailed transaction and parametric data; high 
degree of requirements stability 

The requirements documentation contains Medium - 2 

detailed transaction data; requirements 
modifications limited to pre-PDR 

The requirements documentation is vague; High - 5 

requirements perturbate throughout the total 
development 

27. Does the development employ objective project 4 

control techniques? 

Comprehensive earned value techniques Low - 1 

applied; high degree of management visibility 
into cost and schedule status 

Some earned value methodology applied; Medium - 2 

some management visibility into cost and 
schedule status 

No objective status measurement techniques High - 3 
employed; management visibility based 
primarily on gross resource expenditures 
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STRUCTURE RISK ASSESSMENT 


WEIGHT 


28. Relationships between offices (other than ODP) 

involved with system, i.e., users, customers, sponsors, 
interfaces; those who must be dealt with during 
the project effort 


No significant conflicting needs; serves Low - 1 

primarily one organizational unit 

Meets limited conflicting needs of Medium - 2 

cooperative organizational units 


Must meet- important conflicting needs High - 3 

of several cooperative organizational units 

Must meet important conflicting needs of High - 4 

several uncooperative organizational units 


3 


29. What is severity of procedural changes in user department 3 

caused by proposed system/system enhancement? 

No changes Low - 0 

Minimal changes Low - 1 

Moderate, neither extreme; or unknown Medium - 2 

Significant changes High - 3 


30. Does user organization have to change structurally to 
requirements of new system/system enhancements? 

Minimal 

Low 

Somewhat 

Medium 

Major 

High ■ 
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STRUCTURE RISK ASSESSMENT 


WEIGHT 


31. What is general user attitude? 5 

Good - values data processing solution Low - 1 

Fair - some reluctance Medium - 2 

Poor - does not appreciate data processing High - 3 

solution 


32. How well established are the people, procedures, knowledge, 4 
discipline; and division of details in the offices that (plan 
to) use the system, i.e., is the job the proposed/existing 
system performs well defined and understood? 

Situation satisfactory Low - 1 

Situation satisfactory but could stand Medium - 2 

some improvement 

Situation leaves much to be desired High - 3 


33. Is there a joint developer/user team? 

N/A; project size < 2000 hrs 

Full-time user representation 
and project size > 2000 hrs 

Part-time user representation 
and project size between 2000 - 5000 hrs 

Part-time user representation 
and project size between 5000 - 10000 hrs 

Part-time user representation High - 4 

and project size > 10000 hrs 

No user representation High - 6 

and project size > 2000 hrs 


N/A - 0 
Low - 1 

Meduim - 2 

Medium - 3 


5 
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STRUCTURE RISK ASSESSMENT 


WEIGHT 


34. Commitment of upper-level user management to system 

Extremely enthusiastic Low - 1 
Adequate Medium - 2 
Some reluctance or unknown High - 3 


35. Is project dependent on contribution of technical 2 

effort from other divisions in ODP, e.g. , Systems 
Programming “Division to install new system software? 

no Low - 1 

Yes; from Division(s) within Applications Medium - 2 

Yes; from Division(s) outside of Applications High - 3 
(as well as possibly from those within) 


36. How knowledgeable is user in the field of data processing? 2 

High degree of capability Low - 1 

Previous exposure, but limited Medium - 2 

knowledge 

First exposure High - 3 

37, How knowledgeable is user in proposed application area 2 

(attempt to assess satisfactory use/operation of system 

by user)? 

Previous experience Low - 1 

Conceptual understanding Medium - 2 

Limited High - 4 


38. 


How knowledgeable is project team in proposed 
area? 

Previous experience 
Conceptual understanding 
Limited 


application 


Low - 1 
Medium - 2 
High - 4 


3 
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STRUCTURE RISK ASSESSMENT 


WEIGHT 


39. What degree of control does the project 2 

management have ? 

Formal authority commensurate with assigned Low - 1 
responsibility 

Informal authority commensurate with Medium - 2 

assigned responsibility 

No authority delegated along with High - 3 

responsibility 


40. Are there effective project communications? 

Easy access to project manager (s );. change Low - 1 

information transmitted expeditiously both 
upward and downward 

Limited access to project manager(s); Medium - 2 

downward communication limited 

Aloof project management; planning High - 3 

information closely held 


2 
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STRUCTURE RISK ASSESSMENT 



WEIGHT 

41. 

How well does developed system conform 
to system specifications? 



3 


N/A; new system 

N/A - 

0 



Operational tests indicate actual procedures 
and operations produce desired results 

Low - 

1 



Limited tests indicate that actual Medium - 

procedures and operations differ in only 
minor respects 

2 



Actual procedures and operations differ 
in important respects; specifications 
insufficient to use for testing 

High - 

3 


42. 

Is the project dealing with highly sensitive 
information? 



1 


No 

Low - 

0 



Yes 

High - 

3 


43. 

Does the location of the work require the use of 
specially packaged equipment not currently 
available? 



1 


NO 

Low - 

0 



Yes 

High - 

3 


44. 

Level of clearance required to work on project 



2 


N/A; no problem, project team has 
required clearances 

N/A - 

0 



Need person(s) with low level clearance Medium - 

2 



Need person(s) with high level clearance 

High - 

3 
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STRUCTURE RISK ASSESSMENT WEIGHT 

41. How well does developed system conform 3 

to system specifications? 

N/A; new system N/A - 0 

Operational tests indicate actual procedures Low - 1 
and operations produce desired results 

Limited tests indicate that actual Medium - 2 

procedures and operations differ in only 
minor respects 

Actual procedures and operations differ High - 3 

in important respects; specifications 
insufficient to use for testing 


42. Is the project dealing with highly sensitive, 
compartmented information? 

No 

Yes 


Low - 0 
High - 3 


1 


43. Does the location of the work require the 

use of TEMPEST certified equipment not currently 
available ? 


No 


Low - 0 


Yes 


High - 3 


1 


44. Level of clearance required to work on project 


N/A; no problem, project team has 
required clearances 

Need person (s) with SECRET clearance 
but non-badged ( ISA/S ) 

Need person(s) with TOP SECRET clearance 
and badged ( ISSA/TS ) 


N/A - 0 

Medium - 2 


High - 3 


2 
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TECHNICAL RISK ASSESSMENT 


WEIGHT 


1. Can user fulfill mission during hardware/software failure? 2 

Mission can be accomplished without Low - 1 

system 

Mission can be accomplished without Medium - 2 

fully operational system, but some 
minimum capability required 

Mission cannot be accomplished without High - 6 

fully automated system 


2. What is the required availability of 
the proposed system? 

Periodic use, weekly or less frequent Low - 1 

Required for daily use, but not Medium - 2 

24 hours /day 

Required for 24 hours/day use High - 5 


3. Does proposed/ existing automated system rely on exchange 2 

of data with other external systems, i.e., interfaces, as 
a necessary part of its function? 

Does not require the receipt of data Low - 0 

from another external system to be functional, 
sends no data to other systems required for 
their operation 

Must send/receive data to or from another Medium - 2 
system 

Must send/receive data to or from multiple High - 3 
systems 


4. If proposed/existing system has external interfaces, 
what is the nature of system-to-system communication? 


System has no external interfaces 

Automated communication link 
utilizing standard protocols 

Automated communication link 
utilizing non-standard protocols 


Low - 0 
Medium - 2 

High - 3 


1 
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TECHNICAL RISK ASSESSMENT 

5. What are the size limitations of proposed 
system? 


Substantial unused capacity 

Low 

- i 

Within capacities 

Medium 

- 2 

Pushes capacity near limits 

High 

- 3 


6. How extensive are input data control procedures in 
the system environment? 

# 

Extensive error checking of input data 
Gross error checking 
No error checking 


Low - 1 
Medium - 2 
High - 3 


7. 


What percentage of the current system 
is directly transferable to the proposed system? 


N/A; no current system involved 
50% - 100% 

25% - 50% 

0% - 25% 


N/A 

Low 

Medium 

High 


0 

1 

2 

3 


WEIGHT 

2 


3 


3 


8 . 


What type of system hardware will be 
installed? 

N/A; no hardware to be added 

Standard 0DP batch/ online systems 

Non standard 0DP peripherals 

Non standard ODP peripherals and mainframes 


N/A - 0 
Low - 1 
Medium - 2 
High - 3 


3 
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TECHNICAL RISK ASSESSMENT 


9. What was the basis for the programming and 
system software selections? 

Decision based on architectural Low - 1 

analysis of functional and performance 
requirements 

Decision based "on similar system 
development experience 

Decision based on current inventory 
of system software, and existing 
programming language skills 


10. How complex is the projected system? 

Single function (e.g., word processing only) Low - 1 

Multiple, but related functions (e.g., Medium - 2 

message generation, editing, and 

dissemination) 

Multiple, but not closely related (e.g., High - 3 

data base query, statistical manipulation, 
graphics plotting, text editing) 


11. What level of programming language is 
projected? 

High level in wide usage Low - 1 

Low - 1 level or machine language Medium - 2 

in wide usage 

Special purpose language, extremely High - 3 

limited usage 


Medium - 2 

High - 3 


WEIGHT 

3 


2 
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TECHNICAL RISK ASSESSMENT 


12. How well suited is the programming 
language to the application^ )? 

All modules can be coded in a straight- Low - 1 

forward manner, in the chosen language 

All modules can be coded in a straight- Medium - 2 

forward manner, with few programming 
workarounds required 

A significant number of programming High - 3 

workarounds required, in order to 
compensate for deficiencies in the 
selected language 


13. How familiar is the hardware architecture? 

Mainframe and peripherals widely Low - 1 

used within ODP 

Peripherals unfamiliar Medium - 2 

Mainframe unfamiliar High - 4 

14. Pioneering aspects (extent to which the system applies new, 
difficult, and unproven techniques on a broad scale or in a 
new situation). 

Conservative - No untried system Low - 1 

components, no pioneering system 
objectives or techniques 

Moderate - Few untried systems Medium - 2 

components and their functions are 
moderately important; few, if any 
pioneering system objectives and techniques 

Aggressively pioneering - More than a High - 3 

few relatively untried hardware or 
software components or system objectives 


WEIGHT 

2 


2 
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TECHNICAL RISK ASSESSMENT 


15. 

How well suited is the projected hardware 
to the application environment? 



N/A; standard 0DP hardware being used 

N/A - 0 


Architecture highly compatible with 
required functions 

Low - 1 


Architecture sufficently powerful, but 
not particularly efficient 

Medium - 2 


Architecture dictates complex software 
workarounds 

High - 3 

16. 

What kind of development tools exist ? 



Comprehensive set of automated and 
documented procedural tools available 

Low - 1 


Limited set of automated and documented 
procedural tools available 

Medium - 2 


No tools planned 

High - 3 

17. 

How realistic is the development system? 



N/A; no separate development system 

N/A - 0 


Identical operational and development system 

Low - 1 


Similar operational and development systems 

Medium - 2 

- 

Major architectural differences between 
operational and development systems 

High - 3 


18. Margin of error (necessity for everything to work perfectly, 
for "split-second timing" for great cooperation (including 
external parties), etc.) 

Comfortable margin Low - 1 


Realistically demanding 


Medium - 2 


Very demanding; unrealistic 


High - 3 


WEIGHT 

2 


5 


5 
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TECHNICAL RISK ASSESSMENT 


WEIGHT 


19. Is the application software (e.g., PL1, GIMS, RAMIS, FORTRAN) 2 
new to project team? 

Team is well experienced Low - 1 

Some experience or experience unknown Medium - 2 

Inexperience with programming High - 3 

language or data base 

20. Is the system environment supporting the application 2 

new to the project team? 

( more than one selection may apply ) 


Team is well experienced Low - 1 
Some experience or experience unknown Medi um - 2 
Inexperience with: 

Operating system High - 3 
DBMS High - 3 
Data communications High - 3 


21. How knowledgeable is project team in proposed application 
area? 


Pr e v ious expe r i enc e 

Low 

Conceptual understanding 

Medium 

Limited 

High 


2 


22. What kind of test tools are planned? 

Comprehensive test/debug software 
including path analyzers 

Formal, documented procedural tools 
only 

None 


5 

Low - 1 

Medium - 2 

High - 3 
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TECHNICAL RISK ASSESSMENT WEIGHT 

23. How realistic is the test environment? 4 

Tests performed on operational system Low - 1 

with total data base and communications 

environment 

Tests performed on separate development Medium - 2 

system with total data base, but limited 
communications 

Tests performed on dissimilar development High - 3 

system, with limited data base and limited 
commun ications 


24. How are communication interfaces to. be 4 

tested? 

N/A; no interfaces required 

Live testing on actual line at operational 
transaction rates 

Loop testing on actual line, simulated Medium - 2 

transactions 

Line simulations within development system High - 3 

25. Can critical component testing be performed 2 

with sufficient leadtime to permit redirection? 

Major tests can be performed before all Low - 1 

hardware/software deliveries are 

received 

Only limited testing can be performed Medium - 2 

before all hardware/software 
deliveries are received 

No testing can be performed without all High - 3 

components in place, only simulations 


N/A - 0 
Low - 1 
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TECHNICAL RISK ASSESSMENT WEIGHT 


26. What is the training environment? 1 

Little training needed to use or operate Low - 1 

system, documentation sufficient for 
training purposes 

Users and/or operators can manage Medium - 2 

without formal training, but expertise 
is required in addition to documentation 

Users essentially are unable to use 
system without training, formal, 
hands-on training needed in addition 
to documentation 

27. Is maintenance configuration complex ? 

A single version of one system to 
maintain 

Essentially one user system, but 
training/ development versions must 
be maintained 

Multiple user versions of system in 
operation on different CPUs and/or 
different computer centers 


High - 3 


Low - 1 


Medium - 2 


High - 3 


28. How adaptable is the proposed system to change 3 

High degree, structured programing Low - 1 

techniques used, relatively unpatched, 
well documented. 

Moderate degree Medium - 2 

Low degree, due to monolithic program High - 4 

design, high degree of inter/intra system 
dependency, unstructured development, 
minimal documentation, etc. 
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TECHNICAL RISK ASSESSMENT WEIGHT 

29. What is the nature and type of deliverables 2 

( software, documentation, etc. ) required for 
the project ? 

Relatively small in scope and complexity Low - 1 

and tailored to the needs of the user 
and system maintenance activities 

Determined by selection, based on project Medium - 2 
scope and type, from a standard list of 
well-defined deliverables 

Rigid application of exhaustive deliverable High - 3 

standard, regardless of project scope and 

type 
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- ■ . 


Establishing the Quality 
Assurance Function 


Reviewing Controls in 
Systems under Development 


sponsored by: 

Data Processing 
Management Association 
Education Foundation 

Dedicated to expanding educational 
opportunities for information systems 
professionals 


HtovgTna^Dd-re Fulf-Daxworksliops* 





The Job of the Quality 
Assurance Manager 


Conducting Quality 
Assurance inspections 


March 23-25, 1983 
Chicago, Illinois 


Conference Management by: u.S. Professional Development institute 
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CONCURRENT PRECONFERENCE 
WORKSHOPS 

Wednesday, March 23, 1983 

9:00 a.m.-5:00 p.m. 


conducting 
Quality Assurance inspections > ; ’ M 

*** *Sg 

Workshop icilder: koMrt ibeniu - • -V .Wa4SHuj 
President * '?•- 

Software Mlthodolooy, Inc. — >l 


Establishing the 
Ouality Assurance Function 

This workshop provides the participant with the necessary in- 

d^toKa^Ji^l^T 0 t # he qua,,tv assurance function, it 
on how t0 P^Pare and Implement quality 
standards tandards ’ and how t0 vcr1f v compliance with these 

Workshop leader: william e. Perry 
executive Director 
Quality Assurance institute 



The Job of the ^ ■* 

Ouality Assurance Manager 

The workshop presents: (1) critical success practices of the ai **' 
, strate P , es for leadership In quality imptovS' $ 
^Principles and skHis of managing the MntfnSSt^’ ‘3 
tive. technical, and political responsibilities ahd tasks bfthljBpI^ 

workshop Leader: * N. Schwartz 

General Manager * 

software Quality service, Ihc I 


Reviewing Controls in 
Systems Under Development 

controls by type. Identifying effectiveness and efflden^ 


Workshop leader: 


Ernest a. Relgstad 

Manager, mis Planning ana Policies 
Warner-Lambert Company 


A.M. 


CONFERENCE PROGRAM AND SCHEDULE 

«. ««-, Thursday, March 24 . isaj 

9:00 conference chairman’s opening remarks on: 

The Ten commandments of EDP Ouality Assurance 
wtttiam l Parry 
executive Director 
Quality Assurance institute 

9:1S SKSheSlSS 5 S: Th * * Me£ * 0f 0ata Proee « in 9 Ouality on the Enterprise 
vice President Quality 
IBM Corporation 


10:00 


Fletcher i. luckily* * Pr0ieet ° n $tandard * for software Ouality Assurance 
leee Project Manager 
RCA Corporation 

Roy w e walters art ° f 0ua,,tY ~ ,nst,l,,n Q The Desire for Ouality 
President 

Roy W. waiters a Associates 


12:00 lunch 


P.M. 


ESTABLISHING AND 
BUILDING THE OA FUNCTION 
TRACK 

1:15 Obtaining Support from Systems 
and Programming Personnel for 
Ouality Concepts 

• Impediments to quality 

• Setting quality to analysts and 
programmers 

• Using consultants to sell quality 

• selling senior management 
Stephen a. Bender 

Director of Quality Assurance 
Upstate computer center. Inc 


3.00 


Controlling Changes to 
Applications systems 

• Change control methods 
e Updating documentation 

• Maintaining quality measurements 
e Effect on schedules and budgets 
Mary Kay Holtrop 

Quality Assurance Manager 
valley National Bank 


CONCURRENT SESSIONS 


STRENGTHENING THE 
ESTABLISHED FUNCTION 
TRACK 

Reviewing the Quality of 
System Requirements 
e Quality requirements 
e Establishing review standards 
e How to measure quanty 
e Determining when and where to 


4:00* 

7:00 


Michael i. Fagan 

Manager 

ISM Quality institute 


Having Systems and Programming 
Personnel Conduct Technical 
Reviews 

e Setting review responsibilities 
e Review strategy 
e Selecting a review group 
• Getting the program started 
James M. Jones, IH 
Manager of Systems Development 
McCormick 8 company. Inc 


. t 


ADVANCED QA TOPICS 
TRACK 


Making Reviews Effective 
Change Agents 
e Effective review statements 
e A certification methodology 
• Defining change 
e Cast study example 
Itfward o. JosHn 
Manager 

U.S. Department of Agriculture 


Effective Testing Tools 
and Techniques 
e Setting testing objectives 

• Automating the test process 

• Designing a test plan 

e Overview of testing tools 
Denis C Meredith 
Product Support Manager 
Management and Computer services, i 


QUALITY ASSURANCE SOFTWARE AND SERVICES SHOWCASE 




Sponsor ; 

- AttdcladoA 1 WM' ‘ J 

. ..ItA f # Mvextloil Foundation . *’ : w¥l 

educational oceominltw that wm ad- W^tSJftwSS^^^.*^ 
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